Skip to content

Playwright fix test failures and improve the pytest_runtest_makereport hook #6599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

emilghittasv
Copy link
Collaborator

This pull request includes several changes to improve error handling, and debugging capabilities.

The most important changes include handling the 502 errors for the test_loginless_mozilla_account_aaq test (which was the main flakiness reason for that test), extending timeouts for two messaging system tests (Sometimes SuMo returns the expected search results with a >10 seconds delay so I've bumped the wait to 15) & skipped the AAQ quote test for now .

Error Handling Improvements:

Test Reliability Enhancements:

Timeout Adjustments:

@emilghittasv emilghittasv requested review from Copilot and escattone and removed request for Copilot March 28, 2025 17:52
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR aims to improve error handling and test reliability in the Playwright tests. Key changes include extending timeouts in messaging system tests, handling navigation failures in explore help articles tests by implementing a refresh, and enhancing the pytest hook to properly capture video attachments when tests fail.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
playwright_tests/tests/messaging_system_tests/test_messaging_system.py Increased timeout values to accommodate delayed responses during messaging system searches.
playwright_tests/tests/explore_help_articles_tests/explore_by_topic_tests/test_explore_by_topics.py Added navigation expectation with a refresh fallback when navigation does not occur.
playwright_tests/tests/conftest.py Refactored pytest_runtest_makereport hook for improved video saving and attaching on test failures.
playwright_tests/tests/ask_a_question_tests/aaq_tests/test_posted_questions.py Skipped the quote reply test pending further decision.
playwright_tests/tests/ask_a_question_tests/aaq_tests/test_aaq_form_page.py Added a conditional check to avoid clicking the sign-in button when the auth page is already active.
Comments suppressed due to low confidence (1)

playwright_tests/tests/explore_help_articles_tests/explore_by_topic_tests/test_explore_by_topics.py:43

  • [nitpick] Consider replacing the print statement with a logging mechanism or an Allure step to ensure consistent and integrated reporting of navigation failures.
print("Navigation did not occur. Refreshing the page.")

@emilghittasv emilghittasv merged commit 33ed97e into mozilla:main Mar 31, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants